Many rural areas are characterized by small population centers, which are served via a terrestrial infrastructure with limited capacity, typically copper lines or Digital Microwave Radio (DMR). Although it is possible to substitute or augment the terrestrial capacity through adding a satellite route, the communicating applications have no knowledge or ability to control their forwarding through the different links. Therefore latency sensitive traffic such as Voice over IP may be carried on the high-latency satellite route resulting in poor performance, and latency insensitive file downloads may be carried over the terrestrial route, which is a waste of ‘premium bandwidth’.
Even where a broadband access multiplexer using DSL is available that is appropriately scaled to the size of the potential market, the cost of backhaul from the rural site to and from the network core may be prohibitive. For example, augmenting the terrestrial route could cost millions of dollars, which the potential new revenue cannot possibly justify. The ability to cost effectively augment limited terrestrial networks with satellite would therefore makes it economically practical to extend broadband services to many areas that would not otherwise receive service.
A satellite is a sophisticated electronic communications relay station orbiting the Earth. Earth stations transmit signals to a satellite in orbit over an uplink. Satellites receive this signal, amplify it, shift it to a different (usually lower) frequency and then feed the outgoing signal into an on-board satellite antenna, where the signal is focused into a beam and sent back to Earth along a downlink. Each satellite downlink has a series or set of “beams” (or sectors) illuminating a footprint on the surface of the Earth. A typical satellite might use sixteen such beams. Sometimes multiple beams at different frequencies are used to illuminate the same given area in a single “beam” pattern, with each being referred to as a “sub-beam.” The reception of the satellite signals can either be through a large parabolic antenna at a satellite earth station (e.g. with a 12 meter dish) or received directly at the end customer using a small VSAT (Very Small Aperture ‘Terminal’ dish (eg. typically less than a meter in diameter). It is also possible for the end customer to transmit back to the satellite, although this generally involves larger dishes than for ‘receive only’, and different regulatory conditions such as the need for a certified installation may exist.
In general, satellite-based networks offer certain advantages over terrestrial wireless networks in that they can provide rapid deployment of communication services over a wide geographical area, including remote, rural, urban, and inaccessible areas. They are especially well suited to broadcast transmission of latency insensitive traffic such as broadcast television signals. As an interactive data example, satellite data terminals provide credit and debit card transaction network communications from retail stores, gas stations, and banks at hundreds of thousands of locations around the globe. New locations can be provisioned and decommissioned quickly and at a low cost, compared to the time and cost of connecting locations by other methods. As another example, mobile voice networking via satellites enables individuals to utilize portable computers or handheld satellite phones to connect from remote, low-density population rural areas. Satellite-based networks offer more flexibility in allocating capacity to different sites because the common satellite capacity is shared over what is usually a continental area.
For reception of broadband TV signals at a cable network head-end, parabolic dishes at the earth access station point to each satellite to receive the analog and digital feeds from various content providers utilizing the satellite. The access stations receive, demodulate, decrypt, and decode individual programs; the demodulated (baseband) signals are then remodulated according to a TV standard (NTSC, PAL, SECAM) governing the respective region onto a cable TV frequency plan and combined with local off-air signals and cable modem data to form the broadband service that is amplified and relayed along a coaxial cable plant or hybrid fiber optic and coaxial cable infrastructure to the home subscriber.
The growth in IP traffic and the technical advances in packet technologies have made it possible to support what was once a distinct set of parallel networks (voice, video, wireless), on one integrated data network. This has started a move towards a common packet based network with sharing of common network infrastructure to provide services and interoperability of these services. In addition, there is a trend to enable packet based networks with a new set of feature-rich multimedia communication services, such as integrated messaging, multimedia conversations, on-demand multi-point conference, enhanced security & authentication, various classes of media transport services, numerous automations in electronic Internet commerce activities (banking, shopping, customer care, education, etc.).
Systems have been proposed which make use of satellite communications to access the Internet. For example, Direct TV provides high speed Internet access through a plurality of Direct Broadcast System (DBS) satellites, originally constructed to provide satellite television service. Unfortunately, the Direct TV system does not provide a communication link for the user to send information back to the Internet via the satellite. Accordingly, a computer user must utilize a separate terrestrial telephone line through the PSTN system or the like to provide communications to an Internet provider. Though the transmission rate from the computer user through the PSTN system to an Internet provider is substantially slower than the downlink from the DBS satellite, this system is generally usable where no other form of broadband access is available, as the end users of a DBS system are using internet applications such as web browsing and file transfer that generally receive (download) much greater amounts of data from the Internet than they transmit. Accordingly, it is generally acceptable for the computer user's uplink (transmission) rate to be substantially less than the computer user's download rate.
However, the Direct TV system suffers from several disadvantages as a user must have both a satellite receiver and a connection to telephone service and the performance of the system is fundamentally limited by the speed of the dial-up PSTN link, and more seriously by the latency (delay) in the satellite route. The maximum buffer size in standard TCP/IP implementations of 64 kilobytes combined with a typical satellite delay of 560 ms give a maximum throughput of 936 kbits/sec regardless of what higher speeds the satellite route may theoretically support. The standard Windows receive buffer size is just 8760 bytes, which gives an effective end to end rate of 128 kbit/s or less.
All satellite based communication links have a fundamental latency issue in their performance. The communication satellites to which we refer (excluding Low Earth Orbit) sit in Geostationary orbit about 23,000 miles above the equator, and the delay for a signal to reach the satellite and then return to earth is approximately 500 milliseconds. If the system in question also uses a satellite for the return path from the customer, than the delay exists in both directions (approx. 1 second delay total).
Most Internet Protocol traffic occurs over a TCP layer which assures the integrity of the communication session; it makes the underlying IP transport reliable. TCP uses extensive ‘handshaking’ between the two communicating sites, which unfortunately makes it highly exposed to latency as each of the interactions between the sites must traverse the network.
In downloading a typical web page composed of many individually retrieved elements a unidirectional satellite route nominally described as providing 400 kbit/s downstream throughput may actually load the page slower than a 56 kbit/s dial-up link because the latency in the satellite route causes a delay as each element in the web page is requested, sent, and acknowledged. TCP throughput is fundamentally limited by latency combined with the TCP window size. The TCP/IP protocol incorporates several features which make it work poorly over high latency links, in particular the high number of acknowledgements to verify correct reception of packets, and the ‘slow start’ feature of TCP where the sending rate is gradually increased over time. Most satellite equipment manufacturers have introduced both standard and proprietary modifications to the standard TCP/IP protocol suite in order to minimize these effects, for example by generating using the Internet Engineering Task Force RFC2018 TCP Selective ACK to avoid communications waiting for acknowledgements to traverse the high latency satellite route. Some of the other standardized mechanisms include RFC2488 “Enhancing TCP over Satellite Channels using Standard Mechanisms”, RFC1191 “Path MTU”, and RFC1323 “Large Windows”. Some of these modifications such as TCP protocol spoofing will not work with certain applications; for example IPsec VPN sessions that encrypt the TCP and IP headers. These modifications also have no effect for UDP traffic, which is the basis for most Voice over IP sessions, a key latency sensitive application.
Satellite routes are also generally prone to higher bit error rates than terrestrial routes. The TCP protocol assumes that lost packets are due to congestion in the network, even if bit errors across a satellite route were the real reason. When satellite bit errors occur, TCP will therefore throttle back senders and reduce throughput which degrades service performance and yet has no positive impact on the packet loss rate. TCP will therefore perform better over terrestrial routes for reasons of congestion control mechanisms as well as for latency reasons.
Latency is a critical issue for some but not all applications. Latency is critical for many applications, including Voice, Video-telephony, most online-gaming, Citrix and other thin client applications, Terminal emulators without local echo (e.g. Telnet), PC remote control applications, and online chat. Moderately latency sensitive applications include web browsing and video streaming between a user and their multicast distribution point (due to the need for prompt channel change times). Latency insensitive applications include ftp file transfers, smtp email updates, nntp news updates/Usenet traffic, peer-to-peer file sharing applications, refresh traffic for distributed web caches, broadcast video, and most unicast streaming video.
Voice for example starts degrading substantially in quality when the round trip delay exceeds 200 ms, and special echo cancellation techniques are required. When the voice signal is digitized and transported over Internet Protocol, then latency is even more critical because the encoding and decoding process itself introduces delay.
Many computer systems operating within companies have been programmed with the assumption that the network interconnecting different processing nodes has a known fixed delay, as in fact was true with pre-packet time division multiplexed data networks or in local area terrestrial networks. When the link between two processing nodes in this internal corporate network exceeds the expected latency, it will typically assume that either the network or the remote node has failed and take drastic corrective action such as terminating the session and restarting the associated processing tasks.
General Internet Protocol routers do not examine traffic characteristics to determine which traffic is latency sensitive and do not consider physical link performance characteristics when making forwarding decisions.
The inability of the network to distinguish between the applications it carries is made particularly vivid by recent experience of service providers with peer-to-peer applications such as Kazaa, Morpheus, and Napster. In some parts of the network these applications now constitute over 50% of total traffic volume. They are unusually likely to cause network congestion because they are as likely to be uploading traffic as downloading. Although they are insensitive to latency themselves, they may cause congestion which damages latency and loss sensitive applications.
Current systems deal with latency requirements of the access network by selecting terrestrial or satellite routes according to latency requirements, as for example described in U.S. Pat. No. 6,591,084: “Satellite based data transfer and delivery system” (Chuprun et al.) and related U.S. published Patent application 20030203717: “Satellite based data transfer and delivery system” describe a system for wireless access that has satellite and optionally terrestrial network links. A satellite or terrestrial route is selected based on automatic user node affiliation or user profiles, which may include service cost limitations and allowable delay limitations. As a result, the switching/forwarding decisions are static being based on user profiles and node affiliations and not on real-time inspection of the actual traffic flow. There is a need to provide a system that is more adaptive to changes in traffic patterns and content, for obtaining a more efficient utilization of the satellite and terrestrial network links.
Other prior art solutions that have only satellite network links to access systems for addressing specific requirements. For example, U.S. Patent Application 20030109220: “Communication satellite adaptable links with a ground-based wideband network” (Chuprun et al.) describes basically a land-based network with hub nodes that have satellite routes to remote end-user systems. A communication manager determines the communications requirements between the satellite and the network, and can identify multiple hub nodes for communication with the network if the capacity of a given hub is insufficient to meet the requirements. However, the forwarding of the communication manager is not based on the latency sensitivity of the traffic, and the architecture presents only satellite routes to the end-user systems and not terrestrial ones.
U.S. Patent Application 20030032391: “Low latency handling of transmission control protocol messages in a broadband satellite communications system” (Schweinhart et al.) addresses latency sensitivity of traffic transmitted over satellite communications by queueing and scheduling the traffic for transmission according to its latency limitations. Again, this architecture presents only satellite routes to the end-user systems and not terrestrial ones, so that these systems can not provide any guidance on using alternative types of links, e.g. high-speed terrestrial routes, to reduce latency.
The prior art satellite based access system is a system to give priority to certain users. The invention described is a system to give priority to certain traffic types or applications.
Therefore, viewing the prior art as a whole, an access system for selectively forwarding traffic on satellite and terrestrial routes according to real-time requirements of the traffic and its type is neither disclosed nor suggested. The present invention is directed to such a system.